Affinity in Distributed Systems

In this dissertation we address shortcomings of two important group communication layers, IP Multicast and gossip based message dissemination, both of which have scalability issues when the number of groups grows. We propose a transparent and backward-compatible layer called Dr. Mul-ticast to allow data center administrators to enable IPMC for large numbers of groups without causing stability issues. Dr. Multicast optimizes IPMC resources by grouping together similar groups in terms of membership to minimize redundant transmissions as well as cost of filtering unwanted messages. We then argue that when nodes belong to multiple groups, gossip based communication loses its appealing property of using fixed amount of band-width. We propose a platform called GO (for Gossip Objects) that bounds the node's bandwidth use to a customizable limit, prohibiting applications from joining groups that would cause the limit to be exceeded. Both systems incorporate optimizations that are based on group similarity or affinity. We explore group affinity in real data-sets from social networks and a trace from an industrial setting. We present new models to characterize overlaps between groups, and discuss our results in the context of Dr. Multicast and GO. Ymir Vigfusson became absent from outdoors games with other kids on the block at early age due to " coding projects ". He did a research internship on software vulnerabilities with an Internet security company in Paris in Summer 2001, and a year later he enrolled in Mathematics with Computer Science as a minor at the University of Iceland. He worked for Prof. Magnus Halldorsson at the Icelandic Genomics Corporation in Summer 2004 to design and implement software to estimate binding sites for short primers in-silico. He then worked with Halldorsson on approximation algorithms for sum coloring in interval graphs during the following year. iii To my sisters, Embla and R ¨ oskva. ´ I minningu Bjargar Svavarsdóttur. iv ACKNOWLEDGEMENTS I would first and foremost like to thank my co-authors of the work presented here for the many hours of stimulating, productive and fun conversations. These people are (in alphabetical order) input and assistance, and Mike Spreitzer for producing the WEBSPHERE trace. I feel privileged to have had a chance to work with these great people, and look forward to continue cooperating with them in the future. The same is true for all the other people I worked with on projects that are not included in this dis-sertation. I am deeply …